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ABSTRACT 

The cognitive processes that underlie a medical 
student's improved competence during the clerkship (clinical 
experience) portion of medical education were studied for high- and 
low-achieving medical students in the Netherlands. Nine low-achieving 
and 21 high-achieving fourth-year students and 10 low-achieving and 
14 high-achieving fifth-year students diagnosed two clinical cases 
from written descriptions. Students were retested the following year. 
Analyses suggest that the increased expertise associated with 
advanced study that had been found in earlier studies are not so much 
induced by differences in factual knowledge, but rather by 
qualitative changes in the knowledge base, probably as a result of 
practical experience. Achievement level appears to have no effect on 
the structure measures investigated in this study. There is no 
indication that high and low achievers develop differently or at a 
different pace. Accuracy of diagnosis and explanations given by 
students tend to correlate with achievement level, but effects are 
less strong than those of experience. One table and two figures 
present study findings. (Contains 7 references.) (SLD) 
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Introduction 



Most medical curricula consist of a preclinical and a clinical period. In the preclinical period 
the emphasis is on knowledge and sometimes on skills acquisition as well, while in the 
clinical period students are supposed to learn tc apply their knowledge and skills. These two 
learning activities are often strictly allocated to these two distinct periods. The idea behind this 
stringent division is twofold. One reason is an ethicai one, i.e. that students first must have a 
sound body of knowledge and skills before they can be confronted with patients. The other 
reason has an epistemological background, i.e. that students first must have built up a 
complete and sound body of knowledge before they can learn to apply that knowledge in 
practical situations. 

This dichotomy into different learning situations and activities result in an impetuous change 
in the students' tasks, responsibilities and experiences. During the preclinical period the chief 
activity has been learning from books, lectures and/or small group activities, depending on the 
curriculum structure (traditional or problem-based). Besides they also have had practicals 
giving them hands-on experience with lab procedures, and social, perceptual and 
psychomotor skills trainings preparing them for the requirements of medical practice. During 
the clinical rotations the role and the tasks of the student and the teacher changes completely. 
Now students have to interview and examine real patients all by themselves. They also have 
to show their manual skills such as making stitches, vaccinating a young child, or do some 
small surgery with real patients. Furthermore, they are expected to conform to professional 
standards. They are no longer allowed to behave like students. Instead, they must be 
recognizable as (future) doctors. And although the clinical staff is still responsible for 
diagnosis and treatment, students more than ever have the feeling that something is expected 
from them that really matters, not only to themselves, but also to the patients and to their 
supervisors. Not surprisingly many things change after a medical student has entered 
clerkship. 

Evaluations of the effects of clinical rotations usually focus on the question whether the 
student has become a better doctor, i.e. whether the student has better attitudes than before 
and whether he or she has become more competent. Competence is often defined as the 
ability to apply the relevant knowledge and cognitive, perceptual, psychomotor and social 
skills in order to arrive at an appropriate diagnosis and plan of management and treatment. 
The assessment tools applied, such as OSCEs focus on the skills that are demonstrated in 
such cases. 

The focus of our research is a different one. Our main interest concerns the cognitive 
processes that underlie this competence improvement Earlier research has suggested that the 
development of medical expertise is not only associated with further knowledge and skills 
acquisition, but that a knowledge restructuring process termed 'knowledge encapsulation' and 
illness script formation plays a pivotal role in this process (see Boshuizen & Schmidt, 1992; 



Schmidt & Boshuizen, 1993a). In our theory we assume that especially the practical 
application of biomedical knowledge, which is predominantly learned during the preclinical 
curriculum part, results in knowledge restructuring. This knowledge restructuring has been 
observed in several studies using different paradigms. In one study (Schmidt & Boshuizen, 
1993b) it has been found that medical experts (experienced physicians) use less biomedical 
concepts in explaining the pathophysiological processes than underlie a patient's signs and 
symptoms. On the other hand, the same experts used more concepts that were identical or 
equivalent to the concepts used in a model explanation of the same case than medical students 
of various levels of training applied; contrary to the students they used hardly any concepts 
that are at a more detailed level than the model explanation. This finding was replicated by 
Van de Wiel, Boshuizen and Schmidt (1994), using four new cases. This finding was taken 
to indicate that biomedical knowledge restructures in such a way that detailed biomedical 
concepts encapsulate under 'higher level', clinically relevant concepts. 

Furthermore, in a study in which a think-aloud method and a post hoc explanation task were 
combined (Boshuizen, 1989; Boshuizen & Schmidt, 1992), it was shown that fifth year 
students 2 by the end of their junior clerkship year and medical experts hardly applied any 
biomedical knowledge in clinical reasoning. On the other hand, fourth year students appeared 
to apply elaborate biomedical knowledge when trying to diagnose a case. These observations 
are in line with the findings by Schmidt and Boshuizen (1993b). Fifth year students instead 
tried to diagnose medical cases utilizing clinical knowledge, like experienced physicians. 
They seemed to have activated some illness scripts. However, the illness scripts that these 
fifth year students tried to apply, were incomplete and rather inflexible. Furthermore, the 
experts appeared to apply clinical or disease knowledge in clinical reasoning that integrated 
highly encapsulated biomedical knowledge (or compiled biomedical knowledge as it is called 
in earlier publications). This was not the case in the fifth year students whose biomedical and 
clinical knowledge seemed not to be integrated (Boshuizen, 1989; Boshuizen, Schmidt & 
Coughlin, 1988). 

An explanation for this discontinuity in the restructuring of the knowledge base is suggested 
by research outcomes described by Boshuizen, Hobus, Ousters and Schmidt (1992). In this 
study, subjects with different levels (4) of expertise, ranging from fourth year medical 
students to experienced family physicians were asked to describe prototypical patients having 
a specific disease or to describe its clinical picture. Subjects described 18 prototypes or 
clinical pictures. The results of this study suggested that the intermediate groups had elaborate 
knowledge about the separate components, but that these components were not yei very well 
integrated. This finding suggests that the development of illness scripts and encapsulation of 
biomedical knowledge is not the continuous and interlaced processes as was hypothesized 
before (see Schmidt & Boshuizen, 1993a). 



2 The undergraduate medical curriculum of the University of Limburg consists of a four years preclinical period 
and a clinical period of two years. 



One of the problems with our research designs so far is that a cross sectional approach has 
been taken. Subjects are volunteers, but the different groups are recruited in different ways. 
Preclinical students often put their names on lists in which experimental subjects are asked to 
participate. This is a very clear form of self selection. Students participating in the clinical 
rotations or interns and experienced physicians are approached individually and asked to 
participate. Furthermore, the dangers of a cross-sectional design are, especially in a 
demanding domain like medicine, that the successive groups cannot be assumed to be one 
another's replications. I.e., the groups of experienced physicians (family physicians or 
internists) contained less females than the student groups, while they may also differ in term 
of intelligence, persistence, and so on. Yet, conclusions are made about the longitudinal 
development of medical expertise. 

Such a incompatibility between the groups might have caused some of our findings. E.g., the 
finding that fourth year medical students applied detailed biomedical knowledge while fifth 
year students applied illness scripts (although not very flexible and detailed ones) might be 
caused by differences in amount and quality of the knowledge. This is suggested by the work 
by Lemieux and Bordage (1992), who found that outstanding students' clinical reasoning is 
based on deep, biomedical knowledge, while poor students' clinical reasoning is based on the 
application of defective knowledge or superficial knowledge consisting of unrelated lists of 
facts. Lemieux' and Bordage's findings suggest that the observed differences between fourth 
and fifth year students in our research outcomes might be due to individual differences in the 
two samples. The fourth year students group might have consisted of outstanding students 
applying deep, biomedical knowledge, whereas the fifth year group might have consisted of 
poor or mediocre students, applying superficial knowledge that might be taken for illness 
scripts application. Although we certainly had the impression that both groups consisted of 
rather good, well motivated students, no hard data are available to falsify this hypothesis. 

This paper reports a follow-up study in which self-selection was minimized. Fourth and fifth 
year students were selected, based on a low or a high grade point average on the Maastricht 
progress test and were invited to participate in a one year follow-up study. By combining this 
follow-up and cross sectional approach a period of two years, covering the whole clinical 
period, can be reached. Using groups of high and low scoring subjects enables us to 
investigate the alternative explanation suggested by the Lemieux and Bordage findings. If the 
developmental path of weak students deviates from outstanding students', then high achievers 
might be distinguished from iow achievers with respect to the quality of their diagnoses and 
the quality and organization of the knowledge applied in the case explanation. 



Method 



A combination of a cross-sectional and a follow-up methodology has been applied to 
investigate whether the knowledge development of weak and outstanding students is in the 
same way affected by the clerkships or whether they are differently influenced. Four groups 
c D ® of students were tested and re-tested after a year. A group of high and a group of low 



achieving fourth year students were tested by the end of the pre-clinical period, and one year 
later, halfway the clerkship period. Comparable groups of fifth year students have been tested 
halfway the clerkships and one year later by the end of it, just before or even shortly after 
graduation. Selection of high and low achievers was based on their test performance on the 
Maastricht progress test. A double blind procedure with respect to selection criterion has been 
applied. Furthermore, a reference group of a sample of experienced physicians 
(gynaecologists) has been included. 

Students were asked to diagnose two clinical cases (presented on paper): an anemia case and a 
menstruation disorder case (see appendix 1). After diagnosing each case, the students were 
requested to explain the process that is underlying the patient's signs and symptoms. 
Analysis of expert performance provided us with an anchoring point. Two parallel cases, of 
equal complexity and from the same class of diseases, have been selected. Presentation order 
was balanced across subjects (see appendix 2). 



Subjects and selection 

The scores on two Maastricht progress tests, taken in September and December were used as 
a criterion. These scores were expressed as percentile scores based on the number of correct 
answers 3 . These percentile scores are calculated per year group. As a consequence, the 
fourth-year and the fifth-year student both scoring in the 100th percentile have different raw 
scores, e.g., 48% vs. 58% correct answers. Students with an mean percentile score > 80 or 
< 20 were invited to participate in this study. 

The class of fourth-year students consisted of 167 students. Thirty of them had a mean 
percentile < 20, 25 had a mean percentile score > 80 on the two tests. (This difference might 
indicate that high scores are more persevering than low scores. Probably a low score 
motivates students to work harder, especially in the fourth year where there are less 
possibilities for compensation than in the fifth year.) These people were individually 
approached and urgently requested to participate. Despite this individual approach it was very 
difficult to include low achievers in the study. They were hard to contact, and it was still 
harder to convince them to participate. Even then the experimenters faced a great deal of no- 
shows. Nine of the low achievers and 21 of the high achievers participated. There were 126 
fifth year students. Twenty-seven of them had a mean percentile score < 20, 24 had a mean 
percentile > 80 on the test. Of this group ten of the low achievers and fourteen of the high 
achievers participated. One year later, on the follow-up test these students were again 
approached. Except for three students, everyone could be reached. These numbers are 
summarized in Table 1. 



eric 



3 To prevent undesirable answering strategies student assessment is based on the number of correct answers 
decreased by the number of incorrect answers. Measures based on the number of correct answers are preferred 
for research, because these have better psychometric charateristics. 
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Table 1 



Total number of fourth- and fifth-year students, number selected and number participating in 
the two measurements. 





total group 


selected 


participated; 1st 
measurement 


participated; 2nd 
measurement 


Fourth-year 
students 


167 


£20 N = 30 
> 80 N = 25 


£ 20 N = 9 
2: 80 N = 21 


<, 20 N = 9 
> 80 N = 20 


Fifth-year 
students 


126 


<20 N = 26 
> 80 N = 24 


£ 20 N = 10 
> 80 N = 14 


< 20 N = 9 
>80 N=13 



Analysis 

Quality of the diagnoses and the quality and organization of the knowledge applied in the case 
explanation were determined, applying measures derived from Boshuizen and Schmidt (1992) 
nd Schmidt and Boshuizen (1993b). These studies have shown that level of expertise is 
associated with (a) detail and precision of the diagnosis, and (b) has an inverted U-shaped 
relation with elaborateness of the explanation of the process underlying the patient's signs and 
symptoms: first an increase is observed, followed by a decrease in the clinical years. 
Furthermore, explanations of subjects at higher levels of expertise showed (c) a closer concur- 
rence with a model explanation (explanation and model have an increasing number of concepts 
in common), (d) a decrease in the level of detail of the explanations, (e) an increase in the 
number of model concepts that are omitted in the subjects' explanations, and (f) a greater 
confinement of the subjects' explanation to the concepts applied in the model explanation 
(e.g., subjects restrict themselves to the explanation of relevant signs and symptoms, or do 
not give alternative explanations of the findings). An example of such a model explanation 
can be found in appendix 3. 

Measures c through e are associated with knowledge encapsulation (clustering of detailed, 
biomedical concepts under higher order, often semi-clinical labels), while measure c, 
concurrence with a model also gives an indication of the quality of the knowledge applied. 
The last one, f, can be seen as a result of the increasing coherence of the internal 
representation a subject has built of a case, and is also an indication of the quality of the 
knowledge applied. The better the activated knowledge matches the case description (and the 
model) the more the case can be explained as a whole and the less local explanations for non- 
fitting items have to be constructed, hence less deviations from the model will be observed. 
This situation would occur when a subject has developed a sound knowledge base consisting 
of well differentiated illness scripts. 

Data were analyzed applying level of experience (fourth-, fifth- and sixth-year students) and 
level of achievement (low, intermediate and high) as independent variables, with repeated 
measurement over cases. Three in stead of two achievement levels were compared: (1) 
Students who scored consistently high on the progress tests that were used as a selection 
criterion and on two successive progress tests one year later, (2) students who scored 



consistently low, and (3) students who regressed toward the mean (the intermediate group). 
The last group consisted of ten students: Two low-achieving and four high-achieving fourth- 
year students, and two low-achieving and two high-achieving fifth-year students. 



Results and discussion 

Analysis of (a) the quality of the diagnoses showed a positive linear effect of experience level 
(F(2,96) = 3.159; p = .0469): sixth year students had better diagnoses than fifth year students 
who had better diagnoses than fourth year students. Furthermore, there was a marked ten- 
dency that high achievers had better diagnoses than low achievers (F(2,96) = 2.852; p = 
.0626). The anemia cases turned out to be easier than the menstruation disorder cases, espe- 
cially for fourth- and fifth-year students (F(l,96) = 10.390; p = .0017). Figure 1 shows that 
the performance of the high achieving sixth- year students comes close to the specialists' 
performance. 
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Figure 1. Diagnostic accuracy of subjects of different levels of achievement and experience. 

(b) The elaborateness of the explanations of the process underlying the patient* signs and 
symptoms tended to decrease with increasing experience (F(2,96) = 3.627; p = .0303), coin- 
ciding with the second, negative slope of the inverted U-shaped relation reported in earlier 
studies (Schmidt & Boshuizen, 1993b). Means varied between 16.5 (sixth-year students) and 
22.5 (fourth year students). The elaborateness of the explanations provided by the experts 
(mean = 15.5) could not be distinguished from the sixth year students'. Student-Newman- 
Keuls comparisons only revealed a significant difference between fourth and fifth year 
students (means 22.5 and 16.8 respectively). Achievement level had no effect (F(2,96) = 
1.395; p = .2527). Case had a significant effect (F(l, 96) = 4.995; p = .277): The anemia 
case was more elaborately explained than the menstruation case. 
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The quality and structural aspects of these explanations, and hence of the knowledge applied 
in it, both varied with experience and achievement level, dependent on the measure observed. 
The present analyses suggest that the explanations of students with higher achievement levels 
tend to have (c) more concepts in common with the model explanation than the explanations 
made by students with lower achievement levels (^(2,96) = 2.941; p = .0576), while experi- 
ence has not such effects (F(2,96) = .217; p = .8056). Figure 2 shows that again the high 
achieving, sixth-year students come closest to the specialists. 
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Figure 2. Number of model concepts in the explanations of subjects of different levels of 
experience and achievement 

Furthermore, (d) the number of detailed concepts decreased with increasing experience levels 
(F(2,96) = 3.451;/; = .0357), while achievement level had no effect (F(2,96) = 1.179; p = 
.3120). (e) The number of omitted model concepts in the explanations varied with experience 
level (F (2,96) = 3.649; p = .0297). However, this effect was not as expected: fifth year 
students omitted less concepts than the others, especially in the anemia case. Achievement 
level had no effect (F(2,96) = 1.002; p = .37 1 1). Finally, (f) the explanations of more 
experienced students tended to be more confined to the canonical model. Their explanations 
contained less concepts that were in no way related to that model (F(2,96) = 4.156; p = 
.0186). Achievement level again had no effects (F(2,96) = .439; p = .6458). Table 1 
summarizes these findings. 
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Table 2 

Summary of effects of amount of experience and achievement level on six dependent 

variables 





effects of amount of 
experience 


effects of achievement level 


a. diagnosis 


j) < .05, increase 


.05 < p < .10, increase 


b. elaborateness 


p < .05, decrease 


p>.10 


c. concepts identical to the 
model explanation 


nil 


.05 < p < .10, increase 


d. detailed, biomedical 
concepts 


p < .05, decrease 


p>.10 


e. omitted model concepts 


p < .05, V-shaped 


p> .10 


f. loose concepts 


p < .05, decrease 


p>.10 



These analyses suggest that, in general, the expertise effects found in earlier studies are not so 
much induced by differences in factual knowledge but rather by qualitative changes in the 
knowledge base, probably due to practical experience. The hypothesis that earlier observed 
differences between fourth- and fifth-year students in the application of detailed biomedical 
knowledge might be an effect of differences in achievement levels of the subjects participating 
in that study can be rejected: achievement level has no effect on the structure measures 
investigated in this study. There is no indication that high and low achievers develop 
differendy, or at a different pace. This does not mean that achievement level does not play a 
role in clinical practice: both the accuracy of the diagnoses and explanations tend to correlate 
with it, but the effects are less strong than the experience effects. 
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Appendix 1. 



Example of a case applied 



1. Female, 21 years old, no children 

Profession: secretary 

2. Prior history: Menarche at age 13. Since that time irregular periods every 4 - 6 weeks 

3. Prior history: The duration of the menstrual bleeding is variable. 

In case of pre-menstrual sensations it often takes about 5 days. Otherwise when bleeding 
starts unexpectedly, it can last longer, up till 10 days. 

4. Prior history: Two amenorrheic periods of 4 months 

5. Prior history: appendectomy due to appendicitis at age 16 

6. Setting: She has made an appointment with her family physician a few days ago. Time of 

consultation: 16:30. 

Complaint: Since six months the patient's menstrual cycle has a duration of 8-10 weeks; 
she is somewhat concerned about it. She wonders whether that is normal at her age. 

7. History: Over the last year the patient seldom had pre-menstrual sensations like a tense 

feeling in the breasts or an altered mood; most of the time the menstrual bleeding starts 
completely unexpectedly. If she has had pre-menstrual sensations the menstruation takes 
about 5 days; if unexpectedly bleedings vary between 2 and 10 days. 

8. History: Breast development at age 10. Puberty at age 1 1. First sexual intercourse at age 

19; presently she has a steady sexual relation with one partner. 

9. History: Anticonception: condoms 

10. Physical examination: Slightly increased facial hair grow; whiskers and a slight 
moustache. Some facial acne. 

11. Physical examination: Adipous impression; weight 75 kg, length 1.65 m 

12. Physical examination: BP 125/80 mmHg 

13. Physical examination: Male pubic hair pattern, increased hair growth on the thighs 

14. Physical examination: Mammae: Normal size and consistency 

15. Physical examination: External genitals: no anomalies at labia majora and minora; no 
anomalies at the clitoris 

16. Physical examination: Speculum: vagina no abnormal discharge, nulliparous portio with 
erythroplakia 

17. Physical examination: Bimanual palpation: normal uterus mobile and anteverted. Both 
adnexa somewhat enlarged (estimated diameter 4 - 5 cm) 

18. Lab: at cycle day +30: prolactin normal; LH increased; FSH normal; LH/FSH-ratio 
increased; 17-B-estradiol slightly increased; androstenedione slightly increased; freeT4 
normal; pregnancy test negative 



Appendix 2. 

Presentation order of the four parallel cases at the first and second 
measurement 



Timet 


Time i plus 1 year 


anemia A, menstruation A 


anemia B, menstruation B 


anemia A, menstruation B 


anemia B, menstruation A 


anemia B, menstruation A 


anemia A, menstruation B 


anemia B, menstruation B 


anemia A, menstruation A 


menstruation A, anemia A 


menstruation B anemia R 


menstruation B, anemia A 


menstruation A, anemia B 


menstruation A, anemia B 


menstruation B, anemia A 


menstruation B, anemia B 


menstruation A, anemia A 
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